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A Sharp Restricted Isometry Constant Bound 
of Orthogonal Matching Pursuit 

Qun Mo 


Abstract 


We shall show that if the restricted isometry constant (RIC) of the measurement matrix 

A satisfies 


<Wi(A) < 


1 

7^+T’ 


then the greedy algorithm Orthogonal Matching Pursuit(OMP) will succeed. That is, OMP can recover 
every s-sparse signal x in s iterations from b = Ax. Moreover, we shall show the upper bound of RIC 
is sharp in the following sense. For any given s £ N, we shall construct a matrix A with the RIC 


6 s+ i(A) 


1 

y/s + 1 


such that OMP may not recover some s-sparse signal x in s iterations. 


Index Terms 

Compressed sensing, restricted isometry property, orthogonal matching pursuit, sparse signal recon¬ 
struction. 


I. Introduction 

We shall give a sharp RIC bound of OMP in this paper. First let us review some basic concepts. 
Suppose we have a signal x in E ;V where N is very large. For instance, x represents a digit 
photo of 1024 x 1024 pixels and N = 1024 2 fs 10 6 . Denote ||x||o to be the number of nonzero 
entries of x. We say x is s-sparse if ||x|| 0 < s; and x is sparse if s -C N. In real world, there 
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are many signals are sparse or can be well approximated by sparse signals, either under the 
canonical basis or other special basis/frames. For simplicity, we assume that x is sparse under 
the canonical basis in M jV . 

Suppose we have some linear measurements of a signal x. That is, we have b = Ax where 
A e W nxN and b e R m are given. The key idea of compressed sensing [3], [8] is that it is 
highly possible to retrieve the sparse signal x from those linear measurements, while the number 
of measurements is far less than the dimension of the signal, i.e., m <C N. 

To retrieve such a sparse signal x, a natural method is to solve the following / 0 problem 

min ||o;||o subject to Ax = b (1) 

X 

where A and b are known. To ensure the s-sparse solution is unique, we would like to use the 
restricted isometry property (RIP) which was introduced by Candes and Tao in [4]. A matrix A 
satisfies the RIP of order s with the restricted isometry constant (RIC) 5 S = 5 S {A) if 5 S is the 
smallest constant such that 

(! - < \\ Ax \\l < (! + ll^lll (2) 

holds for all s-sparse signal x. 

If <5 2 s(A) < 1, the l 0 problem has a unique s-sparse solution [4]. The l 0 problem is equivalent 
to the /) minimization problem when 6 2 * (A) < s/2/2, please see [2], [13], [1] and the references 
therein. 

OMP is an efficient greedy algorithm to solve the l 0 problem [7], [16], [17]. To introduce this 
algorithm, we denote An by the matrix A with indices of its columns in Q and we denote xa as 
the similar restriction of some vector x. The following iterative algorithm shows the framework 
of OMP. 

Input: A, b 

Set: O 0 = 0 , r 0 = b, k — 1 
while not converge 

- n k = i U argmax* |(r fe _i, Aei)\ 

- x k = arg min 2 \\A^ k z - b\\ 2 

- r k = b- An k x k 

- k = k + 1 
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end while 

Return x 

Now let us consider the conditions in terms of RIC for OMP to exactly recover any s-sparse 
signal in s iterations. Davenport and Wakin [6] have proven that <5 S+1 {A) < l/(3i/i) is sufficient. 
Later, Liu and Temlyakov [12] have improved the condition to <5 S+1 (A) < l/((y / 2 + l)i/s)• 
Furthermore, Mo and Shen [14] have pushed the sufficient condition to <Wi(^4) < l/(\/s + 1) 
and have shown by detailed examples that a necessary condition is 5 s+ i(A) < 1 j\fs. Later, Yang 
and Hoog [10] improved the sufficient condition to <5 S (A) + >/s 5 S+1 (2L) < 1. 

There is a small gap open between the sufficient condition and the necessary condition of the 
best results. And we would like to fill out the gap in this paper. 

The content of this paper is consisted by two parts. 

• We shall prove that the condition 

<Wi(^4) < T^f 

is sufficient for OMP to exactly recover any s-sparse x in s iterations. 

• For any positive integer s, we shall construct a matrix with c). s4 i = 1 / s + 1 such that 
OMP may fail for at least one s-sparse signal in s iterations. 


II. Preliminaries 

Before going further, let us introduce some notations. For k = 1,2, •• • , N, define e k to be 
the A:-th element in the canonical basis of M' ¥ . That is, is the 1-sparse vector in such that 
only the k -th entry of ej. is 1. For a given matrix A and a given signal x, define 

S k :=(Ax,Ae k ), k = l,... : N. 

Denote S 0 := max fce {i.... jS j \S k \. The following two lemmas are useful in our analysis. 


Lemma II.l. For any s > 0, define t := ±(\As + 1 — 1 )/\fs. Then we have t 2 < 1 and 
\\A(x + te k )\\l-\\A(t 2 x-te k )\\l = 

(1 — t 4 ) (( Ax , Ax) ± yj~s(Ax , Ae k )) \/k — 1, • • • , N. 


( 3 ) 
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Proof: By the definition of t, t 2 = (y/s + 1 — 1 ) 2 /s = (\/s + 1 — 1 )/(\A + 1 + 1) < 1. 
Moreover, by direct expanding and simplification, we have 

\\A(x + te k )\\l - \\A(t 2 x - te k )\\l = (1 - f 4 ) (^{Ax, Ax) + ^ {Ax, Ae k ) 

Now we only need to verify that 2t/(l — t 2 ) = ±y/s. This is indeed true since 

t y/s + l-l 


2 1 

1 -t 2 


= ± 2 - 




7(1- pAL 1 ) = ±75. 
' 75TT±l 


Lemma II.2. If the restricted isometry constant <5 S+ 1 (A) satisfies 

1 


^s+i(^4) < 


( 4 ) 


y/s + 1 

and the support of the signal x is a non-empty subset of { 1, 2,..., s}, then Sq > \S k \ for k > s. 


Proof: Notice this lemma keeps unchanged if we replace x by cx with some non-zero scalar 
c. Therefore, we can assume that ||o:|| 2 = 1. For the given s-sparse x, we obtain 


{Ax, Ax) = ( A x k e k , Ax 
\ k =i 

S 

= ^x k {Ae k , Ax) 

k =1 
s 

^ ^ 7"/,. A k 


k =1 


< -So||x||i 

< S'oV / i||x|| 2 

= VZs 0 . 

Define t := — (y/s + 1 — l)/yfs. Then by the above inequality and Lemma II. 1, we have 

(1 - t 4 )y/s(S 0 - S k ) > (1 - t 4 )((Ax, Ax) - y/s(Ax,Ae k )) 

=\\A(x + te k )\\l-\\A(t 2 x-te k )\\l. 


( 5 ) 
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Moreover, for any k > s, by the definition of <5, + l (/l), we obtain 
\\A(x + te k )\\l-\\A(t 2 x-te k )\\ 2 2 

> (1 - <W^)) (||x||l + t 2 ) - t 2 (1 + <W^)) (t 2 \\x\\l + 1) 
— (1 + 1 2 ) (l — t 2 — (1 + t 2 )5 s+ i(A)) 

= (1 + t 2 ) 2 ((1 -t 2 )/(l + t 2 ) - <s,+i(^)). 


By the definition of t, we have 

(i -t 2 )/(i+( 2 ) = fi 


V s + 1 — 1 \ / A _|_ V s + 1 


Vs + T+1J V V^+T+l 

Now combining (5), (6), (7) and condition (4), we obtain 

(1 — t 4 )Vs(S 0 — Sk) 


= l/Vs + l. 


>\\A(x + te k )\\ 2 2 -\\A(t 2 x-te k )\\l 
>(l+t 2 ) 2 ((l-t 2 )/(l+t 2 )-5 s+1 (A)) 
— (1 + t 2 ) 2 (l /Vs + 1 — <5 s+ i(t4)^ 

> 0. 


( 6 ) 


( 7 ) 


Therefore, we have Sq > Sk for all k > s. Similarly we can prove that S 0 > —Sk for all k > s. 
Hence, S 0 > |.5' fc | for all k > s. ■ 


III. Main Results 

Now we are ready to show the main results of this paper. 

Theorem III.l. Suppose that A satisfies 

<Wi(-4) < / .. > (8) 

Vs + 1 

then for any s-sparse signal x, OMP will recover x from b = Ax in s iterations. 

Proof: Without loss of generality, we assume that the support of a: is a subset of {1,2,..., s}. 
Thus, the sufficient condition for OMP choosing an index from {1,..., s} in the first iteration 
is 

Sq > |S*;| for all k > s. 
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By Lemma II.2, 5 S+1 < guarantees the success of the first iteration of OMP. OMP makes 
an orthogonal projection in each iteration. Hence, it can be proved by induction that OMP always 
selects an index from the support of x in s-iterations. ■ 

Theorem III.2. For any given positive integer s, there exist a s-sparse signal x and a matrix 
A with the restricted isometry constant 


(s+l)x(s+l) 

\ 

' (s+l)x(s+l) 

we can verify that A T Ax = sx/(s + 
1) for all x £ M s+1 satisfying = 0 and Ylk =i Xk = 0- Therefore, s/(s + 1) is an eigenvalue 
of A T A with multiplicity of s — 1. Moreover, by direct calculation, we can see that A T Ax = 
(1±1 /\A + l)x with x = (1,1,..., 1, 1±\A + l) T . Therefore, A T A has another two eigenvalues 
1 ± 1/\/s + 1. Thus, the restricted isometry constant 5 S+1 (A) is equal to ^ +1 . Now let 

x = (1,1,...,1,0) T £ M s+1 . 

We have 

S k = (Ae k , Ax) = ( A r Ae k , x) — s/(s + 1) for all k £ {1,..., s + 1}. 


<WA) = 


s/s + 1 


such that OMP may fail in s iterations. 


Proof: For any given positive integer s, let 

/ 

sj s( s +l) 


A = 






yj s(s+l) 

0 1 


By simple calculation, we get 


A T A = 


( 


—I 

s+l ls 




s+l 


1 

S+l 


1 

s+l 

- 2 - 1 + ^ 

s+l ~ s+l 


where A T denotes the transpose of A. By direct calculation, 
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This implies OMP may fail in the first iteration. Since OMP chooses one index in each iteration, 
we conclude that OMP may fail in s iterations for the given matrix A and the s-sparse signal 
x. ■ 


Remark III.3. For the case of measurements with noise, please see [9], [11], and the references 
therein. 
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